A Multimodal System for Object Learning
نویسندگان
چکیده
A multimodal system for acquiring new objects, updating already known ones, and searching for them is presented. The system is able to learn objects and associate them to speech received from a speech recogniser in a natural and convenient fashion. The learning and retrieval process takes into account information gained from multiple attributes calculated from an image recorded by a standard video camera, from deictic gestures, and from information of a dialog based conversation. Histogram intersection and subgraph matching on segmented color regions are used as attributes.
منابع مشابه
A Multimodal Approach toward Teaching for Transfer: A Case of Team-Teaching in ESAP Writing Courses
This paper presents a detailed examination of learning transfer from an English for Specific Academic Purposes course to authentic discipline-specific writing tasks. To enhance transfer practices, a new approach in planning writing tasks and materials selection was developed. Concerning the conventions of studies in learning transfer that acknowledge different learning preferences, the instruct...
متن کاملDual Track Multimodal Automatic Learning through Human-Robot Interaction
Human beings are constantly improving their cognitive ability via automatic learning from the interaction with the environment. Two important aspects of automatic learning are the visual perception and knowledge acquisition. The fusion of these two aspects is vital for improving the intelligence and interaction performance of robots. Many automatic knowledge extraction and recognition methods h...
متن کاملA Decision Support System for Urban Journey Planning in Multimodal Public Transit Network
The goal of this paper is to develop a Decision Support System (DSS) as a journey planner in complex and large multimodal urban network called Rahyar. Rahyar attempts to identify the most desirable itinerary among all feasible alternatives. The desirability of an itinerary is measured by a disutility function, which is defined as a weighted sum of some criteria. The weight...
متن کاملOnline Object Categorization Using Multimodal Information Autonomously Acquired by a Mobile Robot
In this paper, we propose a robot that acquires multimodal information, i.e., visual, auditory, and haptic information, fully autonomously using its embodiment. We also propose batch and online algorithms for multimodal categorization based on the acquired multimodal information and partial words given by human users. To obtain multimodal information, the robot detects an object on a flat surfa...
متن کاملExtracting Visual Knowledge from the Web with Multimodal Learning
We consider the problem of automatically extracting visual objects from web images. Despite the extraordinary advancement in deep learning, visual object detection remains a challenging task. To overcome the deficiency of pure visual techniques, we propose to make use of meta text surrounding images on the Web for enhanced detection accuracy. In this paper we present a multimodal learning algor...
متن کاملOn the Integration of Grounding Language and Learning Objects
This paper presents a multimodal learning system that can ground spoken names of objects in their physical referents and learn to recognize those objects simultaneously from naturally co-occurring multisensory input. There are two technical problems involved: (1) the correspondence problem in symbol grounding – how to associate words (symbols) with their perceptually grounded meanings from mult...
متن کامل